Manipulator and method for controlling thereof

ABSTRACT

A manipulator is provided. The manipulator includes a gripper, a depth sensor, a force sensor configured to sense an external force acting on the gripper; a memory storing instructions; and a processor configured to execute the instructions to: control the gripper to grasp an object, acquire first information on the object based on a first sensing value obtained by the force sensor while grasping the object, control the gripper such that a first area of the object comes into contact with a surface on which the object is to be placed, acquire location information on a contact area between the first area and the surface, acquire a rotation direction to rotate the object based on the location information and the first information on the object, control the gripper to rotate the object in the rotation direction around the contact area, and based on a second area of the object being in contact with the surface, control the gripper to release the object.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a bypass continuation of International ApplicationNo. PCT/KR2021/016436, filed on Nov. 11, 2021, which is based on andclaims priority to Korean Patent Application No. 10-2020-0167799, filedon Dec. 3, 2020, in the Korean Patent Office and Korean PatentApplication No. 10-2021-0064206, filed on May 18, 2021, in the KoreanPatent Office, the disclosures of which are incorporated by referenceherein in their entireties.

BACKGROUND Field

The disclosure relates to a manipulator and a method for controllingthereof and, more particularly, to a manipulator for seating an objecton the ground and a method for controlling thereof.

Description of Related Art

In recent years, with the development of robot technology, various typesof robots, such as cleaning robots, service robots, industrial robots,or the like are being used. As an example of the industrial robot, thereis a manipulator in the form of a human hand and arm to perform variousoperations.

FIG. 1 is a diagram illustrating a placing operation of a manipulator.Referring to FIG. 1, a manipulator 11 may perform a placing operation ofplacing a grasped object 12 on a ground or surface 13. In this case, themanipulator 11 must stably place an object such that the object does notfall over. In current placing methods, a posture of the object 12 isestimated based on shape information of the object 12, and placement isperformed based on the estimated posture. However, in the currentmethod, the posture of the object 12 cannot be accurately estimated whenthe shape of the object 12 is asymmetric, and thus there is a limitationin a stable placing.

Accordingly, there is a need for a technique for a more stable placingmethod.

SUMMARY

Provided is a manipulator capable of stably placing an object.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

According to an aspect of the disclosure, there is provided amanipulator including: a gripper; a depth sensor; a force sensorconfigured to sense an external force acting on the gripper; a memorystoring instructions; and a processor configured to execute theinstructions to: control the gripper to grasp an object, acquire firstinformation on the object based on a first sensing value obtained by theforce sensor while grasping the object, control the gripper such that afirst area of the object comes into contact with a surface on which theobject is to be placed, acquire location information on a contact areabetween the first area and the surface, acquire a rotation direction torotate the object based on the location information and the firstinformation on the object, control the gripper to rotate the object inthe rotation direction around the contact area, and based on a secondarea of the object being in contact with the surface, control thegripper to release the object.

The first information on the object may include information on gravityacting on the object, and information on a distance from the forcesensor to a line of action of the gravity.

The processor may be further configured to execute the instructions toacquire the location information on the contact area based on the firstinformation on the object, a second sensing value obtained by the depthsensor with respect to the surface, and a third sensing value obtainedby the force sensor while the first area is in contact with the surface.

The processor may be further configured to execute the instructions to:acquire a torque centered on the contact area based on the firstinformation on the object and the location information, and acquire therotation direction based on the torque.

The processor may be further configured to execute the instructions toidentify that the first area comes into contact with the surface basedon the first sensing value obtained by the force sensor being greaterthan a first value.

The processor may be further configured to execute the instructions toidentify that the second area comes into contact with the surface basedon a change amount of the first sensing value obtained by the forcesensor for a predetermined time being greater than a second value.

The processor may be further configured to execute the instructions to:acquire shape information on the object based on a second sensing valueobtained by the depth sensor, and identify a seating surface of theobject including the first area and the second area based on the shapeinformation on the object.

The processor may be further configured to execute the instructions tocontrol the gripper such that the first area comes into contact with thesurface based on a difference between a first direction of a firstnormal vector with respect to the surface and a second direction of asecond normal vector with respect to the seating surface being within apredetermined range.

According to an aspect of the disclosure, there is provided a method forcontrolling a manipulator including a gripper, a depth sensor, and aforce sensor configured to acquire an external force acting on thegripper, the method including: grasping an object with the gripper;acquiring first information on the object based on a first sensing valueobtained by the force sensor while grasping the object; contacting asurface on which the object is to be placed with a first area of theobject; acquiring location information on a contact area between thefirst area and the surface; acquiring a rotation direction to rotate theobject based on the location information and the first information onthe object; rotating the object in the rotation direction around thecontact area; and based on a second area of the object being in contactwith the surface, releasing the object by the gripper.

The first information on the object may include information on gravityacting on the object, and information on a distance from the forcesensor to a line of action of the gravity.

The acquiring the location information on the contact area may includeacquiring location information on the contact area based on the firstinformation on the object, a second sensing value obtained the depthsensor with respect to the surface, and a third sensing value obtainedby the force sensor while the first area is in contact with the surface.

The acquiring the rotation direction may include: acquiring a torquecentered on the contact area based on the first information on theobject and the location information, and acquiring the rotationdirection based on the torque.

The method may further include identifying that the first area comesinto contact with the surface based on the first sensing value of theforce sensor being greater than a first value.

The method may further include identifying that the second area comesinto contact with the surface based on a change amount of the firstsensing value obtained by the force sensor for a predetermined timebeing greater than a second value.

The method may further include: acquiring shape information on theobject based on a sensing value obtained by the depth sensor; andidentifying a seating surface of the object including the first area andthe second area based on the shape information on the object.

According to various embodiments of the disclosure as described above, amanipulator may stably place an object. Accordingly, an accident inwhich the object falls over during the placing process may be prevented.

In addition, effects acquirable or predicted by the embodiments of thedisclosure are to be disclosed directly or implicitly in the detaileddescription of the embodiments of the disclosure. For example, variouseffects predicted according to embodiments of the disclosure will bedisclosed in the detailed description to be described below.

Other aspects, advantages and prominent features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which discloses various embodiments of the disclosure takenin conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a diagram illustrating a placing operation of a manipulator;

FIG. 2 is a diagram illustrating a configuration of a manipulatoraccording to an embodiment;

FIG. 3 is a diagram illustrating information acquired by a manipulatoraccording to an embodiment;

FIG. 4 is a diagram illustrating a method of determining a seatingsurface of an object according to an embodiment;

FIG. 5 is a diagram illustrating a method of acquiring a rotationdirection according to an embodiment;

FIG. 6 is a diagram illustrating a method of placing an object accordingto an embodiment; and

FIG. 7 is a flowchart of a method for controlling a manipulatoraccording to an embodiment.

DETAILED DESCRIPTION

The terms used in example embodiments will be briefly explained, andexample embodiments will be described in greater detail with referenceto the accompanying drawings.

Terms used in the disclosure are selected as general terminologiescurrently widely used in consideration of the configuration andfunctions of the disclosure, but can be different depending on intentionof those skilled in the art, a precedent, appearance of newtechnologies, or the like. Further, in specific cases, terms may bearbitrarily selected. In this case, the meaning of the terms will bedescribed in the description of the corresponding embodiments.Accordingly, the terms used in the description should not necessarily beconstrued as simple names of the terms, but be defined based on meaningsof the terms and overall contents of the disclosure.

The example embodiments may vary, and may be provided in differentexample embodiments. Various example embodiments will be described withreference to accompanying drawings. However, this does not necessarilylimit the scope of the exemplary embodiments to a specific embodimentform. Instead, modifications, equivalents and replacements included inthe disclosed concept and technical scope of this specification may beemployed. While describing exemplary embodiments, if it is determinedthat the specific description regarding a known technology obscures thegist of the disclosure, the specific description is omitted.

The terms such as “first,” “second,” and so on may be used to describe avariety of elements, but the elements should not be limited by theseterms. The terms used herein are solely intended to explain specificexample embodiments, and not to limit the scope of the disclosure.

Singular forms are intended to include plural forms unless the contextclearly indicates otherwise. In the present application, the terms“include” and “comprise” designate the presence of features, numbers,steps, operations, components, elements, or a combination thereof thatare written in the specification, but do not exclude the presence orpossibility of addition of one or more other features, numbers, steps,operations, components, elements, or a combination thereof.

The example embodiments of the disclosure will be described in greaterdetail below in a manner that will be understood by one of ordinaryskill in the art. However, exemplary embodiments may be realized in avariety of different configurations, and not limited to descriptionsprovided herein. Also, well-known functions or constructions are notdescribed in detail since they would obscure the invention withunnecessary detail.

FIG. 2 is a diagram illustrating a configuration of a manipulatoraccording to an embodiment of the disclosure. The manipulator 100 mayinclude a sensor 110, a driver 120, a communication interface 130, amemory 140, and a processor 150.

The sensor 110 may include a depth sensor 111. The depth sensor 111 maysense an object or the ground or surface around the manipulator 100. Theprocessor 150 may acquire an object or shape information (e.g., a pointcloud) of the ground on which the object is to be seated based on asensing value of the depth sensor 111. The depth sensor 111 may includean RGB-D sensor, a light detection and ranging (LiDAR) sensor, and atime-of-flight (ToF) sensor.

The sensor 110 may include a force sensor 112. The force sensor 112 maybe provided in a hand or gripper and a joint included in the manipulator100 to sense an external force acting on the hand and the joint. Forexample, the force sensor 112 may sense gravity acting on an objectgrasped by the hand. That is, the force sensor 112 may sense the weightof the object grasped by the hand. The force sensor 112 may include aforce/torque (F/T) sensor.

The sensor 110 may include an inertial measurement unit (IMU) sensor113. The processor 150 may acquire acceleration information or angularvelocity information of the manipulator 100 based on a sensing value ofthe IMU sensor 113. Also, the sensor 110 may include an RGB camera. Theprocessor 150 may identify an object included in a photographed imagephotographed by the RGB camera. In addition, the sensor 110 may includean encoder for acquiring locations and speeds of joints and linksincluded in the manipulator 100. The encoder may sense the location androtation speed of a motor for driving the joint.

The driver 120 may include an actuator that provides power to themanipulator 100. For example, the actuator may provide torque to thehand and joint. The actuator may include various types of motors, suchas linear motors, alternating current (AC) sub-motors, step motors, orthe like.

The communication interface 130 may include at least one circuit and maycommunicate with various types of external devices according to varioustypes of communication methods. For example, the communication interface130 may acquire information on the ground or surface on which the objectis to be placed from an external device or an external server. Theinformation on the ground may include location and shape of the ground,and direction of a normal vector. The communication interface 130 mayinclude at least one of a Wi-Fi module, a Bluetooth module, a ZigBeemodule, a Beacon module, a cellular communication module, a 3rdgeneration (3G) mobile communication module, 4th generation (4G) mobilecommunication module, a 4G long term evolution (LTE) communicationmodule, and a 5th generation (5G) mobile communication module.

The memory 140 may store an operating system (OS) for controllingoverall operations of the components of the manipulator 100 and commandsor data related to components of the manipulator 100. For example, thememory 140 may store information about the object. The information onthe object may include a magnitude of gravity acting on the object, anda distance (i.e., a moment arm) from the force sensor 112 to an actionline of gravity.

The memory 140 may store data necessary for a module for controlling anoperation of the manipulator 100 to perform various operations. Modulesfor controlling the operation of the manipulator 100 may include anobject information acquisition module 151, a seating surfacedetermination module 152, a ground information acquisition module 153, acontact detection module 154, and a contact information acquisitionmodule 155, a rotation direction acquisition module 156 and a handcontrol module 157. The memory 140 may be implemented as a non-volatilememory (e.g., a hard disk, a solid state drive (SSD), a flash memory), avolatile memory, or the like.

The processor 150 may be electrically connected to the memory 140 tocontrol overall functions and operations of the manipulator 100. When auser command for operating the manipulator 100 is input, the processor150 may load data for modules 151 to 157 stored in the non-volatilememory to perform various operations into the volatile memory. Theloading refers to an operation of loading and storing data stored in thenon-volatile memory into the volatile memory such that it can beaccessed by the processor 150.

The object information acquisition module 151 may acquire variousinformation about the object. For example, the object informationacquisition module 151 may acquire shape information of an object basedon a sensing value of the depth sensor 111. The shape information of theobject may include a point cloud corresponding to the object.

As another example, the object information acquisition module 151 mayacquire a weight of the object grasped by the manipulator 100. Theobject information acquisition module 151 may acquire gravity (i.e., aweight of an object) acting on the object based on a sensing value ofthe force sensor 112 acquired in a state in which the manipulator 100 isstopped while grasping the object.

As another example, the object information acquisition module 151 mayacquire a first moment arm corresponding to the object. The first momentarm may refer to a distance from the force sensor 112 to a line ofaction of gravity acting on the object in a state where the object andthe ground do not contact each other. The object information acquisitionmodule 151 may acquire the first moment arm based on the force andtorque sensed by the force sensor 112. For example, the objectinformation acquisition module 151 may acquire the first moment arm(h_(g) ^(s)) corresponding to the object based on Equation (1).

$\begin{matrix}{h_{g}^{z} = \frac{f_{g}^{z} \times \tau_{g}^{s}}{{f_{g}^{s}}^{2}}} & (1)\end{matrix}$

where each of f_(g) ^(s) and τ_(g) ^(s) refers to force and torquesensed by the force sensor 112 in a state that the hand grasps theobject.

The information about the acquired object may be stored in the memory140.

The seating surface determination module 152 may determine a seatingsurface of the object to come into contact with the ground on which theobject is to be placed. The seating surface determination module 152 maygenerate a plurality of convex hulls for an object based on a pointcloud corresponding to the object. The convex hull may refer to avirtual two-dimensional plane in a form of a polygon including a partialarea of an object. The seating surface determination module 152 mayidentify one convex hull including a seating surface among a pluralityof convex hulls. For example, the seating surface determination module152 may identify a convex hull in which a minimum distance from a pointcorresponding to a center of gravity of an object to an edge of theconvex hull is maximum among a plurality of convex hulls. A moredetailed description of the method of identifying the seating surfacewill be described below with reference to FIG. 4

The ground information acquisition module 153 may acquire information onthe ground. Particularly, the ground information acquisition module 153may acquire a normal vector corresponding to the ground. The groundinformation acquisition module 153 may acquire a point cloudcorresponding to the ground based on a sensing value of the depth sensor111. In addition, the ground information acquisition module 153 mayacquire an average normal vector corresponding to the ground based onthe point cloud.

Herein, the ground may refer to an arbitrary surface on which an objectis placed, and may include a surface of a workbench or table on which anobject can be placed as well as a ground in the dictionary meaning. Inaddition, the ground may have a flat shape, but is not limited theretoand may have a curved shape.

The contact detection module 154 may detect a contact between the objectand the ground. For example, the contact detection module 154 mayidentify whether the object comes into contact with the ground based ona value sensed by the force sensor 112. When a value acquired bysubtracting a weight of the object stored in the memory 140 from amagnitude of a force sensed by the force sensor 112 is greater than apredetermined first value, the contact detection module 154 may identifythat the object comes into contact with the ground. Alternatively, thecontact detection module 154 may identify that the object comes intocontact with the ground by comparing the magnitude of the force sensedby the force sensor 112 with a predetermined value.

In addition, the contact detection module 154 may identify whether theobject comes into contact with the ground based on the amount of changein the value sensed by the force sensor 112. For example, if the amountof change in the value sensed by the force sensor 112 sensed for apredetermined time is greater than a predetermined second value, thecontact detection module 154 may identify that the object comes intocontact with the ground.

A contact detection method of the contact detection module 154 may varydepending on a contact state between the object and the ground. Forexample, when the object and the ground do not contact (i.e., a firststate), the contact detection module 154 may identify whether the objectcomes into contact with the ground by comparing a value obtained bysubtracting the weight of the object stored in the memory 140 from amagnitude of the force sensed by the force sensor 112 with apredetermined first value. When a first area of the object comes intocontact with the ground (i.e., a second state), the contact detectionmodule 154 may identify whether the second area of the object comes intocontact with the ground by comparing the amount of change in the sensedvalue of the force sensor 112 with a predetermined second value.

The contact detection method using the force sensor 112 has beendescribed in the above, but this is only an embodiment, and the contactdetection module 154 may identify whether the object comes into contactwith the ground based on a value sensed by the IMU sensor 113. Forexample, when a magnitude of a signal sensed by the IMU sensor 113 isgreater than a predetermined value, the contact detection module 154 mayidentify that the object comes into contact with the ground.

The contact information acquisition module 155 may acquire locationinformation about a contact area between the object and the ground. Thecontact information acquisition module 155 may acquire a second momentarm (h_(c) ^(s)), which is a distance from the force sensor 112 to aline of action of an external force by contact. For example, the contactinformation acquisition module 155 may acquire the second moment arm(h_(c) ^(s)) based on a value sensed by the force sensor 112 sensed whenthe object comes into contact with the ground. The contact informationacquisition module 155 may calculate the second moment arm (h_(c) ^(s))based on Equations (2) and (3).

$\begin{matrix}{{f_{c}^{z} = {{f^{z}(t)} - f_{g}^{z}}},{r_{c}^{z} = {{r^{z}(t)} - r_{g}^{s}}}} & (2) \\{h_{c}^{s} = \frac{f_{c}^{s} \times \tau_{c}^{s}}{{f_{c}^{s}}^{2}}} & (3)\end{matrix}$

f^(s)(t) and τ^(s)(t) refers to a sensing value (i.e., force/torque) ofthe force sensor 112 sensed when the object comes into contact with theground, and also f_(g) ^(s) and τ_(g) ^(s) refer to a sensing valueacquired before the contact of object and the ground. In other words,f_(g) ^(s) and τ_(g) ^(s) refer to information on an object stored inthe memory 140.

The contact information acquisition module 155 may acquire an externalforce (f_(c) ^(s), τ_(c) ^(s)) caused by the contact between the objectand the ground by subtracting a sensing value) before the object comesinto contact with the ground from a sensing value (f^(s)(t), τ^(s)(t))when the object comes into contact with the ground. Also, the contactinformation acquisition module 155 may acquire a second moment arm(h_(c) ^(s)) based on the external force (f_(c) ^(s), τ_(c) ^(s)) by thecontact.

The contact information acquisition module 155 may acquire a location ofa contact area) based on the external force (f_(c) ^(s), τ_(c) ^(s)) bythe contact, information on the object (f_(g) ^(s), τ_(g) ^(s)), and thesensing value of the depth sensor 111. The contact informationacquisition module 155 may acquire a location (τ_(c) ^(s)) of thecontact area based on Equation (4) and (5).

$\begin{matrix}\left\{ \begin{matrix}{r_{c}^{s} = {h_{c}^{s} + {af}_{c}^{s}}} \\{{{n^{s} \cdot r_{c}^{s}} + b^{s}} = 0}\end{matrix} \right. & (4) \\{r_{c}^{s} = {h_{c}^{s} - {\frac{b^{z} + {n^{s} \cdot h_{c}^{s}}}{n^{s} \cdot f_{c}^{z}}f_{c}^{s}}}} & (5)\end{matrix}$

The equation (n^(s)·r_(c) ^(s)+b^(z)=0) indicates a plane equationcorresponding to the ground. The contact information acquisition module155 may acquire a point cloud corresponding to the ground based on thesensing value of the depth sensor 111. In addition, the contactinformation acquisition module 155 may acquire a plane equation (i.e.,n^(s)·r_(c) ^(s)·b^(z)=0) based on a point cloud and a plane extractionalgorithm.

The rotation direction acquisition module 156 may acquire a rotationdirection for rotating the object based on the contact area as arotation center. For example, the rotation direction acquisition module156 may acquire the rotation direction based on the information (f_(g)^(s), τ_(g) ^(s)) on the object and the location (r_(c) ^(s)) of thecontact area. The rotation direction acquisition module 156 may acquirethe rotation direction ({circumflex over (ω)}_(rot) ^(s)) based onEquations (6) and (7).

$\begin{matrix}{\tau_{cg}^{s} = {\left( {h_{g}^{s} - r_{g}^{s}} \right) \times f_{g}^{s}}} & (6) \\{{\hat{\omega}}_{rot}^{s}\mspace{14mu}\text{:=}\mspace{14mu}\frac{\tau_{cg}^{s}}{\tau_{cg}^{s}}} & (7)\end{matrix}$

The rotation direction acquisition module 156 may acquire a torque(τ_(cg) ^(s)) around the contact area as a center of rotation based on avector (h_(g) ^(s)−r_(c) ^(s)) from the contact area between the objectand the ground to a line of action of gravity and gravity (f_(g) ^(s))acting on the object. Also, the rotation direction acquisition module156 may acquire a rotation direction ({circumflex over (ω)}_(rot) ^(s))based on the acquired torque (τ_(cg) ^(s)).

The hand control module 157 may generate a control command forcontrolling a hand included in the manipulator 100 and may control anoperation of the hand based on the control command. For example, thehand control module 157 may control the hand to grasp the object. Also,the hand control module 157 may control the hand to move or rotate whilegrasping the object. In this case, when a difference between a firstdirection of a first normal vector with respect to a seating surface ofthe object and a second direction of a second normal vector with respectto the ground is within a predetermined range, the hand control module157 may control the hand such that the object comes into contact withthe ground. Accordingly, the first area of the object may contact theground. In addition, the hand control module 157 may rotate the objectin a rotation direction ({circumflex over (ω)}_(rot) ^(s)) with thefirst area of the object and the contact area of the ground as arotation center. Accordingly, the second area of the object may comeinto contact with the ground. If it is identified that the second areaof the object comes into contact with the ground, the hand controlmodule 157 may control the hand to release the grasp of the object.

FIG. 3 is a diagram illustrating information acquired by a manipulatoraccording to an embodiment of the disclosure. Referring to FIG. 3, themanipulator 100 may acquire a point cloud corresponding to an object 31based on a sensing value of the depth sensor 111. Also, the manipulator100 may acquire a force (f_(g) ^(s)) and a torque (τ_(g) ^(s)) acting onthe object 31 based on a sensing value of the force sensor 112. Also,the manipulator 100 may acquire a first moment arm (h_(g) ^(s))corresponding to the object based on the Equation (1) described above.The first moment arm (h_(g) ^(s)) means a distance from the force sensor112 to a line of action L of gravity. The line of action L of gravitymay pass through a center of gravity C of the object 31.

FIG. 4 is a diagram illustrating a method of determining a seatingsurface of an object according to an embodiment of the disclosure. Themanipulator 100 may determine a seating surface 42 to come into contactwith the ground among a plurality of surfaces of an object 41. Themanipulator 100 may generate a plurality of convex hulls with respect tothe object 41 based on a point cloud corresponding to the object 41. Themanipulator 100, for each of the plurality of convex hulls, maycalculate gravity vector v passing through a center of the point cloudand a distance from an intersection P of the convex hull to a side ofthe convex hull (d_(si)), and a distance (d_(pi)) from the intersectionP to a vertex. The manipulator 100 may determine a convex hull where aminimum value (d_(min, A)) of a distance from the intersection P to theside/vertex of the convex hull is a maximum convex hull, among aplurality of convex hulls, as a seating value A^(s).

The manipulator 100 may move the object 41 such that the determinedseating surface 42 comes into contact with the ground. In this case, themanipulator 100 may move the object 41 such that a difference betweenthe first direction of the first normal vector with respect to theground and the second direction of the second normal vector with respectto the seating surface 42 is within a predetermined range. Accordingly,as shown in FIG. 5, a first area A1 of an object 51 may come intocontact with a ground 52.

FIG. 5 is a diagram illustrating a method for acquiring a rotationdirection according to an embodiment of the disclosure. The manipulator100 may rotate the object 51 around the first area A1 that is a contactarea between the object 51 and the ground 52. The manipulator 100 mayacquire a rotation direction to rotate the object 51. For example, themanipulator 100 may acquire a torque (τ_(cg) ^(s)) with the contact areaA1 as the rotation center, based on a vector (h_(g) ^(s)−r_(c) ^(s))from the contact area A1 of the object 51 and the ground 52 to the lineof action L of gravity and gravity (f_(g) ^(s)) acting on the object 51.The manipulator 100 may acquire the torque (′ based on Equations (6) and(7) described above. In addition, the manipulator 100 may acquire arotation direction ({circumflex over (ω)}_(rot) ^(s)) based on theacquired torque (τ_(cg) ^(s)).

FIG. 6 is a diagram illustrating a method of placing an object accordingto an embodiment. As shown in FIG. 6, the manipulator 100 may rotate anobject 61 based on the rotation direction ({circumflex over (ω)}_(rot)^(s)) Accordingly, a second area A2 of the object 61 may come intocontact with a ground 62. The manipulator 100 may identify that thesecond area A2 comes into contact with the ground 62 when the amount ofchange in a sensing value of the force sensor 112 for a predeterminedtime is greater than a predetermined second value. In addition, themanipulator 100 may release a grasp of the object 61. Accordingly, theobject 61 may be stably placed on the ground 62.

FIG. 7 is a flowchart of a method for controlling a manipulatoraccording to an embodiment of the disclosure.

In operation S710, the manipulator 100 may grasp an object by using ahand.

In operation S720, the manipulator 100 may acquire and store informationon the object based on a sensing value of the force sensor acquiredwhile grasping the object. In this case, the manipulator 100 may acquirelocation information on a contact area based on the stored objectinformation, a sensing value of a depth sensor with respect to theground, and a sensing value of the force sensor acquired when the groundcomes into contact with the first area.

In operation S730, the manipulator 100 may bring the first area of theobject into contact with the ground. In this case, the manipulator 100may bring the first area of the object into contact with ground suchthat a difference between a first direction of a first normal vectorwith respect to the ground and a second direction of a second normalvector with respect to a seating surface is within a predeterminedrange. The seating surface of the object may be acquired based on shapeinformation of an object acquired based on the sensing value of thedepth sensor. When the sensing value of the force sensor is greater thana first predetermined value, the manipulator 100 may identify that thefirst area comes into contact with the ground.

In operation S740, the manipulator 100 may acquire location informationon the contact area between the first area and the ground. Themanipulator 100 may acquire location information on a contact area basedon the object information, the sensing value of the depth sensor withrespect to the ground, and the sensing value of the force sensoracquired when the ground comes into contact with the first area.

In operation S750, the manipulator 100 may acquire a rotation directionfor rotating the object based on the location information and theinformation on the object.

In operation S760, the manipulator 100 may rotate the object in theacquired rotation direction. In this case, the manipulator 100 mayrotate the object around the contact area as a rotation center.

In operation S770, when a second area of the object comes into contactwith the ground depending on the rotation of the object, the manipulator100 may release the grasp of the object. When the amount of change inthe sensing value of the force sensor for a predetermined time isgreater than a predetermined second value, the manipulator 100 mayidentify that the second area comes into contact with the ground.

Various exemplary embodiments described above may be embodied in arecording medium that may be read by a computer or a similar apparatusto the computer by using software, hardware, or a combination thereof.In some cases, embodiments described herein may be implemented by theprocessor itself. In a software configuration, various embodimentsdescribed in the specification such as a procedure and a function may beembodied as separate software modules. The software modules mayrespectively perform one or more functions and operations described inthe present specification.

Methods of controlling a display apparatus according to variousexemplary embodiments may be stored on a non-transitory readable medium.When the computer instructions stored in such a non-transitorycomputer-readable medium are executed by a processor, a specificapparatus may perform a processing operation according to variousembodiments described above.

The non-transitory computer readable recording medium refers to a mediumthat stores data and that can be read by devices. For example, thenon-transitory computer-readable medium may be a compact disc (CD), adigital versatile disc (DVD), a hard disc, Blu-ray disc, universalserial bus (USB), a memory card, a read-only memory (ROM), or the like.

The foregoing exemplary embodiments and advantages are merely exemplaryand are not to be construed as limiting the disclosure. The presentteaching can be readily applied to other types of apparatuses. Also, thedescription of the exemplary embodiments is intended to be illustrative,and not to limit the scope of the claims, and many alternatives,modifications, and variations will be apparent to those skilled in theart.

What is claimed is:
 1. A manipulator comprising: a gripper; a depthsensor; a force sensor configured to sense an external force acting onthe gripper; a memory storing instructions; and a processor configuredto execute the instructions to: control the gripper to grasp an object,acquire first information on the object based on a first sensing valueobtained by the force sensor while grasping the object, control thegripper such that a first area of the object comes into contact with asurface on which the object is to be placed, acquire locationinformation on a contact area between the first area and the surface,acquire a rotation direction to rotate the object based on the locationinformation and the first information on the object, control the gripperto rotate the object in the rotation direction around the contact area,and based on a second area of the object being in contact with thesurface, control the gripper to release the object.
 2. The manipulatorof claim 1, wherein the first information on the object comprisesinformation on gravity acting on the object, and information on adistance from the force sensor to a line of action of the gravity. 3.The manipulator of claim 1, wherein the processor is further configuredto execute the instructions to acquire the location information on thecontact area based on the first information on the object, a secondsensing value obtained by the depth sensor with respect to the surface,and a third sensing value obtained by the force sensor while the firstarea is in contact with the surface.
 4. The manipulator of claim 1,wherein the processor is further configured to execute the instructionsto: acquire a torque centered on the contact area based on the firstinformation on the object and the location information, and acquire therotation direction based on the torque.
 5. The manipulator of claim 1,wherein the processor is further configured to execute the instructionsto identify that the first area comes into contact with the surfacebased on the first sensing value obtained by the force sensor beinggreater than a first value.
 6. The manipulator of claim 1, wherein theprocessor is further configured to execute the instructions to identifythat the second area comes into contact with the surface based on achange amount of the first sensing value obtained by the force sensorfor a predetermined time being greater than a second value.
 7. Themanipulator of claim 1, wherein the processor is further configured toexecute the instructions to: acquire shape information on the objectbased on a second sensing value obtained by the depth sensor, andidentify a seating surface of the object including the first area andthe second area based on the shape information on the object.
 8. Themanipulator of claim 7, wherein the processor is further configured toexecute the instructions to control the gripper such that the first areacomes into contact with the surface based on a difference between afirst direction of a first normal vector with respect to the surface anda second direction of a second normal vector with respect to the seatingsurface being within a predetermined range.
 9. A method for controllinga manipulator comprising a gripper, a depth sensor, and a force sensorconfigured to acquire an external force acting on the gripper, themethod comprising: grasping an object with the gripper; acquiring firstinformation on the object based on a first sensing value obtained by theforce sensor while grasping the object; contacting a surface on whichthe object is to be placed with a first area of the object; acquiringlocation information on a contact area between the first area and thesurface; acquiring a rotation direction to rotate the object based onthe location information and the first information on the object;rotating the object in the rotation direction around the contact area;and based on a second area of the object being in contact with thesurface, releasing the object by the gripper.
 10. The method of claim 9,wherein the first information on the object comprises information ongravity acting on the object, and information on a distance from theforce sensor to a line of action of the gravity.
 11. The method of claim9, wherein the acquiring the location information on the contact areacomprises acquiring location information on the contact area based onthe first information on the object, a second sensing value obtained thedepth sensor with respect to the surface, and a third sensing valueobtained by the force sensor while the first area is in contact with thesurface.
 12. The method of claim 9, wherein the acquiring the rotationdirection comprises: acquiring a torque centered on the contact areabased on the first information on the object and the locationinformation, and acquiring the rotation direction based on the torque.13. The method of claim 9, further comprising identifying that the firstarea comes into contact with the surface based on the first sensingvalue of the force sensor being greater than a first value.
 14. Themethod of claim 9, further comprising identifying that the second areacomes into contact with the surface based on a change amount of thefirst sensing value obtained by the force sensor for a predeterminedtime being greater than a second value.
 15. The method of claim 9,further comprising: acquiring shape information on the object based on asensing value obtained by the depth sensor; and identifying a seatingsurface of the object including the first area and the second area basedon the shape information on the object.
 16. An device comprising: agripper; a memory storing instructions, and a processor configured toexecute the instructions to: acquire first information on an objectbeing grasped by the gripper based on a force sensing value; acquirelocation information on a contact area between a first area of theobject and a surface with which the object comes into contact; acquire arotation direction to rotate the object based on the locationinformation and the first information on the object; rotate the objectin the rotation direction around the contact area; and release theobject by the gripper based on a second area of the object being incontact with the ground.
 17. The device of claim 16, wherein theprocessor is further configured to execute the instructions to: acquirea torque based on the first information on the object and the locationinformation, and acquire the rotation direction based on the torque. 18.The device of claim 16, wherein the processor is further configured toexecute the instructions to identify that the first area comes intocontact with the surface based on the force sensing value being greaterthan a predetermined first value.
 19. The device of claim 16, whereinthe processor is further configured to execute the instructions toidentify that the second area comes into contact with the surface basedon a change amount of the force sensing value for a predetermined timebeing greater than a predetermined second value.
 20. The device of claim16, wherein the processor is further configured to execute theinstructions to: acquire shape information on the object based on adepth sensing value, and identify a seating surface of the objectincluding the first area and the second area based on the shapeinformation on the object.